*» i 



;’;*‘ : boCDMENT RESUME 

PS 005 634 

*’ ' J . « 

les L,. ; And "Others 

Evaluation of the Appalachia Preschool 
^ * r '* / V. Education Program. 

^fs^ITukaON :; "^' Appalachia ‘Educational Lab,, charleston, w. Va. 
bIreAUJ^O 6-2909 

' De c 7 1 

50 p. ; Summary report 




NOTE”, 1 . 



*4 



EDRS, PRICE ' 
DESCRIPTORS 



* -p 

» 



IDENTIFIERS 



MF-10.65 HC-$3* 29 

Cognitive Objectives ; *cornparative Analysis; ♦Early 
Childhood Education; Educational Finance; 
♦Educational ’Progrh|ns; Educational Television; Field 
studies; Home ‘.Visits; individual Development; 
Interpersonal; Competence; Kindergarten Children; 
Language Development; Mobile classrooms; Preschool 
Children; ♦preschool Education; ♦Program Evaluation; 
Psychomotor Skills; Task Performance; Testing; Test 
Results 

♦Appalachia Preschool Education Program 



ABSTRACT 

A report of the findings from the three-year field 
test of the Appalachia Preschool Education Program (APEPj , an early 
childhobd education program, is presented. The report describes the 
human and fiscal effort required to operate the program and the 
changes in the performances of children resulting from the use of the 
program. It also contains a description of the target population for 
the program. Findings include; (1) scores on a test of cognitive 
objectives favored children who received the Appalachia Preschool 
Education Program; (2) A trend toward increased language development 
was observed for APEP children as opposed to children comparison 
groups; (3) Children who participated in the program scored 
significantly higher than comparison groups in psychomotor skills; 

(4) Children who participated in the mobile classroom activities gave 
indications of having developed more constructive social skills than 
children who received only the home visitor and television program; 
and (5) Children in the program reached significantly more cognitive 
objectives than children who received no treatment, (Author/CK) 
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Foreword 

The Appalachia Educational Laboratory defines educational development as 
the systematic process' of creating and diffusing alternative products that 
vnll contribute to the improvement of educational practices. The Appalachia 
Preschool Education Program has been developed by the Laboratory as an alter- 
native to other forms of preschool education, and summative evaluation was 
conducted to indicate possible behavior changes and cost efficiency through 
' use of the product. 

This Summative Evaluation Report is based on three years of field testing in 
southern West Virginia, from 1968 through 1971, The report contains a sum- 
mary section of conclusions concerning the performance of children who par- 
ticipated in the Appalachia Preschool Education Program as well as resources 
required for operating the program. The summary section is based on a series 
of 10 technical reports which give a statistical background for the conclu- 
sions stated in the summary report. All technical reports are available from 
the Laboratory, and they, along with the summary section, may be found in the 
Educational Resources Information Center (ERIC) microfiche system. In order 
to reduce bias regarding the evaluation of a product, the summative evalua- 
tion activities at AEL are the responsibility of a staff division separate 
from the program staff. 

The Laboratory is grateful for the consultative support provided to the sum- 
mative evaluation throughout the three years of field testing. Acknowledg- 
ment is especially extended to Dr, Frank Hooper of the University of Wisconsin, 
Dr, John Kennedy of the Ohio State University, Dr. Charles Kenoyer of West 
Virginia University, and Dr. Ray Norris of George Peabody College, Dr. James 
Runson of the West Virginia University Kanawha Valley Graduate Center was 
Director of Research and Evaluation during the first field test year, and 
Dr, Roy Alford was Director of the Appalachia Preschool Education Program 
during the three, years of field testing. Dr. John Seyfarth of the West 
Virginia Kanawha Valley Graduate Center assisted with preparation of the 
reports for publication. 



Benjamin E. Carmichael, Director 
Appalachia Educational Laboratory, Inc, 
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SummaHve Evaluation of the Appalachia 
Preschool Education Program 

introduction 

This is a report of the findings from the three-year field test of the 
Appalachia Preschool Education Program, an early childhood education program 
which has been developed by the Appalachia Educational Laboratory , Inc, The 
report describes -the human and fiscal effort required to operate the Appa- 
lachia Preschool Education Program (APEP) and the changes in the performances 
of children resulting from use of the program. The report also contains a 
description of the target population for the program, The v niajor findings 
of the three years of field testing are summarized on pages 28 through 30 of 
this report. 

Immediately following is a brief description of AEL’s Appalachia Pre- 
school Education Program and the plan which was used for evaluating the 
program, * 

The Program 

The Appalachia Preschool Education Program is a home-oriented instruc- 
tional system designed for 3-, 4-, and 5-year-old children. It consists of 
30-minute television lessons broadcast into the home each day* a weekly home 
visit by a paraprofessional who discusses the program with parents and chil- 
dren and who delivers materials used by themj and group instruction pro- 
vided once each week in a mobile- classroom taken near the home for convenience 
to parents and small children, 

i 

The program is based on behavioral objectives which were developed by 
West' Virginia University (Hooper and Marshall, 1968, p, 97-197) from a 
nationwide study of preschool education programs and an assessment of pre- 
school Appalachian children, A materials development team was employed to 

* ' 
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translate those objectives in, to television lessons, materials for home use 
by parents and children, and materials and exercises for use in group instruc- 
tion in the mobile classroom. 

The television lessons, recorded on videotape, were produced in Charleston 
W.Va. They were sent to Oak Hill, W.Va. , where they were broadcast by a 
commercial television station over an eight-county area of southern West 
Virginia. The home visitation and mobile classroom components of the program 
operated out of the field test headquarters at Beck ley, W.Va. Eight para- 
professionals were employed and trained to perform home visitation services, 
and one regularly certified teacher and an aide were employed to operate and 
furnish mobile classroom instruction. The fully equipped S' x 22' classroom 
was mounted on a two and one-half ton truck chassis. Power for the operation 
of the heating and cooling system and all electronic equipment in the class- 
room was provided through metered outlets mounted on poles at each scheduled 
stop of the classroom. 

The Laboratory has produced several publications which give a more 
complete description of the Appalachia Preschool Education Program. A series 
of guides to parallel this summative evaluation report will give comprehen- 
sive descriptions of the different components of the program and detailed 
specifications for establishing the program. These guides will be available 
by June 1, 1972, Attachment 1 contains a list of available and proposed 
Laboratory publications which describe various aspects of the program. 



The Plan for Evaluation 

Summative evaluation is considered to be an integral part of educational 
development at the Appalachia Educational Laboratory. The Laboratory s Model 
for Educational Development is presented as Attachment 2 to the summary report. 
The model includes stages which permit an educational product to be taken from 
an initial Needs Assessment through Feasibility Analysis and Program Planning 
to the Design and Engineering stage. The product in this case is the Appa- 
lachia Preschool Education Program which is now completing the fifth, or 
Field Testing, stage. Following Field Testing is a stage called Operational 
Testing during which the Laboratory exercises fewer controls; at that time, 
the product, if found successful, is. disseminated to and implemented by the 
target population* 

Summative evaluation is of primary importance during the Field Testing 
stage since future use of the product by both the Laboratory and the target 
population is dependent on evidence that the product has met objectives 
established during the Design and Engineering stage. 

The Appalachia Preschool Education Program was field tested for three 

years (1968-1971), during which data were collected for the purpose of surnma 
tive evaluation. An evaluation report was produced at the end of each of the 



first two years (see Attachment 2), In practice, the decision to enter the 
Operational Testing stage with the program was based in part on a rather 
positive evaluation following the second year of field testing. 

The plan for summative evaluation of the program was to collect) analyze 
and report data which would indicate: the effort required to produce an 

Appalachia Preschool Education Program, the performance of individuals who 
received the program, and the characteristics of the population which is 
expected to receive the program. The required effort is based on records 
of personnel employed, materials used, and expenses incurred. 

The statements about the performance of individuals who received the 
program are based on standardized test data, data from measures of the 
achievement of specific objectives, altitudinal and interest measures, and 
socioeconomic data. The specific measures and the frequency with which they 
were taken are described under the section of the report entitled Program 
Performance. A description of the population to be served by the program is 
given under the section entitled Program Pervasiveness, and the findings of 
this summary evaluation report are further summarized in the section entitled 
Evaluation Synthesis. 

The field test results are presented by four major categories in order 
to facilitate educational decision-making as suggested by Stufflebeam (1971). 
The Program Effort and Program Pervasiveness sections of the report, although 
based on data from approximately 600 children in the Beckiey field test, are 
presented in terms of a population unit of 25,000 children. The Appalachia 
Preschool Education Program was designed as a regional program and cannot be 
feasibly produced and operated for two or three hundred children. For 
example, the total operational , cost for preparing the materials and video- 
tapes totaled $204,410, and the amount would vary only slightly according to 
the number of children who watched the tapes and used the materials. This 
cost would be unreasonable if prorated over a few hundred children, but would 
average only $8.18 per child if the APEP were used for 25,000 children on a 
regional basis. Other costs, such as that for paraprofessional home visitors 
vary directly with the number of children served regardless of the scale of 
operation. 

Program Effort 

Program effort is defined operationally as time, personnel, and money 
required to acquire, install, operate and maintain an operational Appalachia 
Preschool Education Program serving 25,000 children. Program effort is 
categorized by four major functions* acquisition and installation of 
facilities and equipment, operational requirements for the field test, equip- 
ment and facility maintenance requirements , and program cost analysis * 



v r .. 



Acquisition and Installation of Equipment and Facilities, ; 



Information pertaining to acquisition , and instaliation of equipment is 
presented in Table 1, As indicated in the fable, most equipment required 
for the television component of the program Was rented. Studio time and 
office space for the materials team were available in a commercial studio 
in Charleston, W.Va, Certain technical personnel also were included in the 
studio package. Office space for the field Operation of mobile classroom 



personnel and home visitors was available .in/Beckley , W.Va. 



Parking spaces for the mobile classroom were secured from churches, 
schools, and community centers. Power companies installed ten 220-volt 
meters for operation of the mobile classroom. The coordinator of the field 
testing operations made arrangements for these facilities. Experience indi- 
cated that at least one year of lead time is needed for making these arrange- 
ments and preparing for the production of the. television lessons. No major 
legal obstacles were encountered to prevent .the installation of the field 
testing operation. * - 



Special consultants were used in connection with acquisition and installa 
tion of some equipment. Some services were; required to establish specifica- 
tions for media requirements, and extensive; services were required to develop 
specifications for the mobile classroom facility. 

A publication scheduled for completion by the APEP Development Team by 
June 1, 1972, will give a more detailed description of the equipment and 
facilities needed to operate the program ('se?/ Attachment 1]. 

\ \«.ii 

• * X. « 

Operational Requirements *j!’ 

.1 

A second section of the evaluation p 1 a n- p e r t. a i n e d to the program effort 
needed to meet operational requirements, including personnel, time expended 
by personnel, requirements from other participating agencies, personnel 
training, and formative evaluation. Formative evaluation was conducted to 
determine whether program components and specific activities were meeting 
the objectives for which they were designed^ op a day-to-day basis. Data 
recorded on operational requirements for 25,000 children are presented in 
the first column of Table 2. These estimate's., 'are based on data recorded 
during the three-year field test cycle and are' basically the actual costs 
of developing the program projected onto a population of 25,000 children. 

* & ^ 4 ^ \ • 

A staff of eight professional personnel with an average salary of 
$13,820 is suggested for the preparation of, materials and TV lessons. The 
titles of these eight professionals, which wptiPd compose a. curriculum 
materials team, might be: Curriculum Materials Coordinator, Production 

Manager, Curriculum .Specialist (2), Graphic^Afts Specialist, On-Camera 
Teacher, Artist-Photographer, and an Instructional Monitor responsible for 
formative evaluation. Two small and two large broadcast stations are listed 











Table 1 



Acquisition and Installation of Equipment and Facilities 






Program Component 


Equipment and Facilities 


Method of Acquisition 


Television 


Studio package; Two black and 
white cameras, film chain and 
requisite components for control 
room, lighting, sound and taping. 


Rental 




Darkroom 


Rental 




Four 16mm cameras 


Purchased 




Office and workroom space 


Rental 




Transmitting station 


Rental 


Home Visitation 


Field office 


Rented 




Automobiles 


Personal cars 


Mobile Classroom 


Preschool classroom, 8’ x 22', 
mounted on two and one-half ton 
International truck chassis. 
Classroom includes electric 
heating and air conditioning, 
carpeted floors, six listening 
stations, record player, 16mm 
projector, overhead projector, 
projection screen, psychedelic 
lights, hot plate stove, 
refrigerator, restroom facilities, 
and storage cabinets. 


Purchased 




Ten power supply meters 


Contracted 




Parking locations 


Donated 




Field Office 


Rented 




( 5 ) 





Table 2 



Projected Cost of the Appalachia Preschool Education Program 
for 25,000 Children for One Year* 



1, Preparation of Materials and TV Lessons 

A, Personnel 

8 professional personnel average $13,820 each $110 

3 support personnel average $5,000 each 15 

Staff travel average $550 each 4 

Consultants (i nc. travel) 1 

B, Office and Studio Facilities 



Production $100 per hour, 2 hr. per program 
(170 programs) 

Equipment repair and maintenance 

C. Production Supplies 

Videotapes (replacement) 

(50 tapes i $75/tape) 

Movie and still film (inc. processing) 

Other (set materials, studio art supplies, etc.) 

D. Broadcast Facilities (2 large and 2 small stations) 
Small Stations 

(2 stations, $30 per broadcast, 170 broadcasts) 
Large Stations 

(2 stations, $50 per broadcast, 170 broadcasts) 

E 

E. Capital Outlay 
Equipment 

Videotapes (340 i $75) 

Total cost of preparation of materials and lessons 

Operation 
Per Pupil 
Capital outlay 
Per Pupil 



34 



3 

4 
3 



10 

17 



12 

25 



204 

37 



*Based on actual APEP field test costs prorated to 25,000 children except 
as noted. 
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560 

000 

400 

500 



000 

750 



750 

000 

250 



200 

000 



,000 

,500 



,410 

3.18 

,500 

.50 



Table 2 (Continued) 



II. Field Operation** 

A. Personnel , 

Supervisory 

20 supervisory personnel 

(18 @ $12,000, 2 i $16,000) 

11 support personnel, av. $5,000 
Travel ($1,250 each, 20 persons) 

Field Staff 

167 teachers, av. $9,600 each*** 

(one per 150 children for mobile classroom) 
167 aides, av. $3,500 (for mobile classroom) 

667 home visitors, av. $3,500 
(one per 37.5 children) 

Travel for home visitors 

($600 per H.V. , 667 H.V's.) 

Consultants (pre— and inservice) 

$1,500 each office, 9 offices 

B. Rent, Communications, Utilities 

Office rental ($^,600 per year each, 10 offices) 
Furniture rental ($584 each office, 10 offices) 
Utilities, custodial ($840 each, 10 offices) 
Telephone ($608 each office, 10 offices) 

C. Children's Supplies 

Books, modeling clay, etc. 

($2,200 per unit of 150 children, 167 units) 



$248,000 

55.000 

25.000 



1,603,200 

584,500 

2,334,500 



400,200 



13,500 



46,000 

5,840 

8,400 

6,080 



367,400 



**Based on nine field offices, each responsible for approximately 2,800 
children, and one central administrative office. 

***Figure based on West Virginia estimated average teacher's salary of 
$7,458 (Rankings of the Counties, 1970, West Virginia Education 
Association, April, 1970, p. 14) for 10 months, or $8,950 for 12 months 
plus an additional $650 for driving the mobile classroom. 
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Table 2 (Continued) 



D, Equipment Repair, Maintenance and Insurance 

Repair, Maintenance, gasoline (167 mobile classrooms, 
$1,400 each) 

Insurance and Bonds 

(167 mobile classrooms, $733 each) 

E. Capital Outlay 

Mobile Classrooms (167 units, $16,000 each)**** 



Total field costs 

Operation 
Per Pupil 
Capital outlay 
Per Pupil 



Total Program Costs (I § II) 

Operation 
Per Pupil 
Capital outlay 
Per Pupil 

Capital outlay amortized over five years 
Per Pupil 






Actual cost of mobile classroom to AEL was $21,000 for one uni 
A reduced cost of $16,000 would be incurred for multiple units 






$233,800 



122,411 



2,672,000 



6,053,831 

242,15 

2,672,000 

106.88 



6,258,241 

250.33 

2,709,500 

108.38 

541,900 

21.68 



t. 



under Broadcast Facilities (I-D, Table 2], These stations would enable the 
TV lessons to be seen over an area about the size of the state of West 
Virginia. -s 

I 

An estimate based on field test results indicated that 20 supervisory 
personnel would be needed for the field test operation (II, Table 2). Two 
professionals would be located in each of nine field offices, and two pro- 
fessionals would be located in one central office. One professional in 
each office might be generally responsible for curriculum development and 
the other professional might assmne responsibility for the field operation. 
These field personnel would share responsibility for recruiting teachers 
and paraprofessionals , preservice and inservice training of personnel, and 
distribution of materials and instructions. Personnel requirements for 
25,000 children also include 167 certified teachers, 167 aides, and 667 
paraprofessional home visitors. In addition, consultants would be needed 
for preservice and inservice training. 

These personnel requirements are based on a population unit of 25,000 
children, either approximately all the same age or spread equally among 
ages 3, 4, and 5 years. The configuration of field offices might vary 
according to geographic spread of the population, ■ but the total personnel 
requirements and expenses would remairi approximately the same. 



Maintenance 

The effort required to maintain the equipment and facilities for opera 
tion of the program should be of interest to those considering operating an 
early childhood education program. Equipment associated with the TV lesson 
production was maintained as a part of the lease with the TV studio where 
programs were produced and the station where the programs were transmitted. 
Minor equipment such as movie cameras was maintained by the curriculum 
materials team photographer. 

Mobile classroom facility maintenance was covered by an agreement with 
the local dealer from which the equipment was purchased. Terms specified 
that the equipment be made available to the dealer one afternoon per week 
for the purpose of routine maintenance checks and repair as needed. The 
dealer also provided an on-call emergency service in case of breakdown. 

(In the West Virginia program this was a Guardian Maintenance Agreement 
through Raleigh Motors, the local International dealer.) Maintenance of 
the media equipment and other specialized equipment in the mobile classroom 
was the responsibility of the teacher. 



Program Cost Analysis r 

The cost for the Appalachia Preschool Education Program for a popula- 
tion unit of 25,000 during one year is also given in Table 2. The cost is 



based on the actual experience during the three field test years which are 
further described in Technical Report, No, 11. ' Following is an explanation 
of the projected cost for 25,000 children. 

The average salary of $15,820 for the eight professional personnel on 
the Curriculum Materials Team [I-A, Table 2) was their average annual salary 
during 1969-70, The cost of support personnel was judged to be at about the 
same rate per person as with the Appalachia Preschool Education Program, but 
staff travel was thought ';o be somewhat less than for the Curriculum Materials 
Team since much of the team’s travel was due to the developmental nature of 
the program. The cost of office and studio facilities is considered average 
for this type of operation, and there is the possibility that at least some 
color videotapes could be produced at a cost of $100 per hour. Other costs 
for preparation of materials and TV lessons are based on two years’ experi- 
ence in operating the program. The 340 videotapes listed under Capital 
Outlay would permit one program tape to be shared by two stations, thereby 
creating the necessity of transporting program tapes between stations. An 
additional cost of $25,500 would permit simultaneous broadcasting. 

Total operational cost for preparation of materials and television 
lessons was projected to be $204,410 or $8.18 per child when prorated over 
25,000 children. The capital outlay required for preparation of materials 
and television lessons was projected to be $37,500 or $1,50 per child. 
Alternative funding arrangements could cause office and studio facilities, 
certain production supplies, and/or broadcast facilities to be included as 
capital outlay rather than operational cost. The above cost estimates could 
vary considerably according to the type of videotapes and materials desired. 

The cost for field operation (II, Table 2} is presented separately 
because field operation costs vary more directly with the number of children 
who are to be served. The 18 field supervisors are estimated to require 
$12,000 each, and the two central office supervisory personnel are estimated 
at $16,000 each. Annual travel for these personnel is estimated at $1,250 
each, since their duties include supervising teachers and home visitors over 
an area with approximately 2,800 children. The $9,600 average salary for 
professional teachers is explained in the footnote to Table 2|^and the aides 
and home visitors are expected to require $3,500 each.. Their salaries, as 
well as other costs listed in Table 2, are based on actual field test 
expenses. The consultants are to be used for preservice and inservice 
training of teachers, aides, and home visitors. The $122,411 expenditure 
for insurance and bonds may be funded through some alternative arrangement. 

The cost for office and furniture rental may be considered as capital outlay 
under some funding arrangements. The total operational cost for field opera- 
tions was projected to be $6,053,831 for 25,000 children or $242.15 per child. 
The capital outlay for field operations was $2,672,000, the cost of 167 mobile 
classrooms. 

The total projected cost of operation of the Appalachia Preschool Educa- 
tion Program for 25,000 students for one year was $6,^58,241 or $250,33 per 



child. The total capital outlay for the program for 25,000 children was pro- 
jected at $2,709,500. If equipment is amortized over a five-year period, 
the cost per child for capital outlay is $21.68. 

A separate study was undertaken to determine the comparative cost of a 
standard kindergarten program in West Virginia as operated in 1969-70 (Pre- 
school for Appalachia, AEL) . Based on statistics provided by the West 
Virginia Department of Education, the per pupil cost of operation for a 
full day kindergarten program was $496 (compared to $250,33 for APEP) , and 
the capital outlay costs for a standard kindergarten were found to be more 
th an 7,5 times greater than for the AEL program. 

The fact that the Appalachia Preschool Education Program can be pro- 
duced for about one half the cost of standard kindergarten programs was 
considered important to the evaluation of the program. If the children 
participating in the program were found to reach objectives appropriate 
for 3-, 4-, and 5-year-old children, then the procedures developed by the 
Laboratory would be most acceptable. The following section examines the 
degree to which improved performance did result from the program. 

Program Performance 

Program performance was operationally defined as learning which occurred 
in the target population — 3-, 4-, and 5-year-old children--as a result of the 
Appalachia Preschool Education Program, as measured by selected instruments 
and procedures. Learning was classified into cognition, language, psycho- 
motor, social skills, affective, and interest categories. The first three 
categories were used for conceptualizing the original behavioral objectives 
for tne program. The social skills and affective categories were added as 
technology and funds became available. 

Measurement of social skills development was first attempted during 
the 1969-70 field test year, and the procedure was replicated during the 
final field test year. Due to a lack of established procedures, affective 
learning by children was only indirectly measured. Data on children's and 
parents' interest were collected throughout the three-year field test cycle. 

The following instruments and methods were employed to measure aspects 
of performance considered important to the success of ; the Appalachia Pre- 
school Education Program. Language was defined operationally as responses 
to the Illinois Test of Psyeholinguistic Abilities (ITPA) , Cognition was 
defined operationally as responses to the Peabody Picture Vocabulary Test 
(PPVT), and responses to the Appalachia Preschool Test, a picture test 
similar in format to the PPVT and ITPA. Intelligence was included in the 
category of cognition. Psychomotor development was measured by scores on 
the Marianne Frostig Developmental Test of Visual Perception, and the social 
skills achievement of children was measured by a specially designed inter- 
action analysis technique. Interest was defined operationally as responses 
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to attitude checklists developed by AEL staff and responses re fleeted* in 
anecdotal records systematically collected during the year. 



Three Years of Evaluation. In general, the evaluation of the first field 
test year failed to show substantial gains for the Appalachia Preschool 
Education Program (AEL, 1970), Children's and parents* interest was high, 
but especially the children participating in the mobile classroom activites 
failed to achieve pronounced cognitive gains. 

By the end of the second field test year, the evaluation indicated that 
gains on all measures were becoming more pronounced, with children who 
received home visitors and the mobile classroom scoring above their peers 
who only watched the TV program or received no treatment. However, the chil- 
dren who visited the mobile classroom still failed to show greater gains in 
the cognitive, language, and psychomotor areas than their counterparts who 
received only the home visitor and TV program, so the decision was made to 
increase the children’s exposure on the mobile classroom from 1,5 hours per 
week to two hours per week. Also, because of a resignation, a different 
mobile classroom teacher was employed beginning with the third field test 
year. Although the mobile classroom primarily was designed to improve social 
skills, the combination of increased time, a different teacher, and possibly 
a cumulative growth only observable in the final year tended to give the 
children who attended the mobile classroom increased performance in certain 
language skills. The increase in cognition, language, and psychomotor skills 
was not considered great enough to warrant the expense of the mobile class- 
room but, as explained in the following sections, gains in social skills and 
children's curiosity were attributed to participation in the group activities 
of the mobile unit. } 

The following section is a description of the children who participated 
in the various treatment groups and comparison groups during the third field 
test year (1970-71) of the Appalachia Preschool Education Program. 

Sampling Procedures. The evaluation design used to measure program perfor- 
mance ^f"^i - ^paTachi a Preschool Education Program utilized three treatment 
groups located in Raleigh and Fayette counties in south central West Virginia 
The initial sample was selected in 1968 by randomly assigning treatments to 
3-, 4-, and 5-year-old children living within randomly selected geographic 
grids in the rural areas. Additional children were added each year as some 
of the sample became old enough to enter the public schools. During the 
third year of testing (September 1970 -June 1971), approximately 300 children, 
aged 3, 4, and 5 were enrolled in the program. The number of boys and girls 
enrolled was about equal, as were the relative sizes of the three treatment 
groups . 

One of these groups (TV-HV-MC) received visits from the mobile class- 
room, as well as the paraprofessional home visitor, and watched the 




television program Around the Bend , The second of the groups (TV-HV) watched 
the program and was visited by the paraprofessional , while the third (TV only) 
received only the television program. The sample sizes for different groups 
of children are given in Table 3, 

In September, 1970, a control group of 60 children was identified in 
Monongalia* and Upshur counties, in north central West Virginia, This sample 
consisted of equal numbers of boylh and girls, who were 3, 4, or 5 years of 
age as of that month. An additional 60 children were sampled from the same 
area, and all 120 control groUp_children were tested in June of 1971. This 
was done to provide a (modified) Solomon four-group experimental design which 
permitted a check on the effect of repeated testing. Selection and testing 
of the control samples was done by the West Virginia University Human 
Resources Research Institute, 

Prior to the third year of field testing (1970-71) , school personnel in 
the Beckley, W.Va. , field test area requested that achievement of children 
in kindergarten programs be compared with that of children in the Appalachia 
Preschool Education Program. Sixty-six children in two public school kinder- 
gartens were therefore pre and post-tested during the third field test year. 

The control group was selected from an area that was demographically 
similar to the Beckley area, and the kindergarten group was located in the 
same area. Pertinent socioeconomic data are presented in Technical Report 
No, 11* but it should be pointed out here that these individuals closely 
resemble the overall population of the state in regard to level of income 
and education. 

Mean scores from the Hollingshead Two Factor Index of Social Position 
in each group are reported in Table 4, The differences in socioeconomic 
level as measured by the Hollingshead were not statistically significant, 
and the ordered mean scores, from highest socioeconomic level to lowest 
level, were control, kindergarten, TV-HV, TV-HV-MC , and TV only. Further 
description and comparison can be found in Technical Reports No, 11 and 23. 

In June, 1970, all children enrolled in the second year's program 
effort were given the following test battery* Sub tests 2 and 3 of the 
Marianne Frostig Test of Perceptual Development, Subtest 5 of the Illinois 
Test of Psycholinguistic Ability (ITPA) , Part 2 of Appalachia Preschool 
Test (APT, curriculum specific test) , and the Peabody Picture Vocabulary 
Test (PPVT) . These post-test measures for the second year's 3- and 4 -year- 
olds also served as the pretest for the same children as they became 4 and 
5 years old and entered into the third year's programming. 

Subsequently, it was decided to administer all remaining subtests of 
the above cited battery in September, 1970, to those children who had not 
taken them previously. In that month, 3-year-olds just entering the program 
also received the entire test battery. At the same time a sample of 66 
children attending two kindergartens in the program area were tested with 
the APT and PPVT. 



Table 3 

Number of Children In Treatment 
Groups by Age 



Age 


TV-HV-MC 


TV-HV 


TV only 


Control 


Kindergarten 


3 


25 


. 34 


13 


36 


— 


4 


39 


47 


22 


34 


— 


5 


31 


49 


31 


33 


66 


Total 


95 


130 


66 


103 


66 



Table 4 

Mean Ho llingshead Socioeconomic 
Scores by Treatment Group* 



— 


; tv-hv-mc 


TV-HV 


TV only 


Control 


Kindergarten 




3.74 


3.70 


4.07 


3.49 


3.64 



‘Higher scores indicate lower socioeconomic levels. The first four groups 
include data from 3-, 4-, and 5-year-old children, and the kindergarten 
group was composed of only 5-year-olds, , 







Due to these three -mo nth fluctuations in pre to post-test interval, 
pre to post-test intervals for 4- and 5-year-olds in the three APEP groups 
are one year for the following: PPVT Raw Score, IQ, and MA, ITPA 5, Prostig 
2 , and Prostig 3. Other subtests ( i . e . , ITPA 1-4 and 6-10, Prostig 1, 4, 
and 5 , and APT 5 and 6) for the 4- and 5-year-olds represent intervals of 
nine months. All pre to post-test intervals for 3-year-olds in all treatment 
groups, all ages of the control group, and for the kindergarten group repre- 
sent elapsed time from September, 1970, until June, 1971, a period of nine 
months. The testing during the second and third field test years was com- 
pleted by testers trained by the staff of AEL's Division of Research and 
Evaluation, and it is strongly suggested that home visitors not do the 
testing because they know and are known by the children. 

The following section of this summary report describes changes in 
performance observed by those participating in the Appalachia Preschool 
Education Program. The categories under which the effects of the AEL 
program will be described are: cognitive growth, language development, 

psychomotor, and combination of test results. These will be followed by: 
social skills development, parental interest in television programs, and 
comparison with kindergarten programs . 

Cognitive Growth. Cognition was defined as the ability of a child to recog- 
nize numbers and symbols correctly and to make associations. During the 
first year of the program (1968-69), the Appalachia Preschool Test (APT) was 
designed to measure the cognitive objectives of the APEP. Additional objec- 
tives were emphasized during the second and third years of the field test, 
and representative items were added to the APT. 

As the test was revised, certain sub tests or parts were deleted and 
others added. The subtests used during the final year of field testing were 
Parts 1, 2, 5, and 6, Part 1 was an 11-item interview, and Parts 2 and 6 
were each 61-item subtests which sampled program objectives taught toward 
during the three years of field testing. Part 5 of the APT contained 18 
items measuring logical reasoning, sensory discrimination and labeling, and 
letter recognition. 

The items on the different subtests were taken from program objectives, 
which were in turn taken from objectives derived from a study of Appalachian 
preschoo.L-children as well as an examination of preschool intervention pro- 
grams available at that time (Hooper and Marshall, 1968) . The results 
obtained from these subtests were, therefore, considered most important to 
the evaluation of program success. 

As indicated in Table 5, all subtests of the curriculum specific measure 
(APT) administered near the conclusion of the three-year field test showed 
significant treatment effects favoring groups of children who viewed the 
television program and were routinely visited by paraprofessionals . Although 
the order of means reported in Table 5 from highest to lowest is generally 
TV-HV-MG, TV-HV, TV only, and control, the parapxofessional home visitor 
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Table 5 * - 

Mean Post-test Scores and Levels of Significance 
for the Appalachia Preschool Test 
by Treatment Group* 



Sub test 


TV-HV-MC 


TV-HV 


TV only 


Control 


Significance** 


1 


8.2 


8.1 


7. 6 


7.6 


<.001 


2 


40.7 


38.6 


35.3 


29,9 


<.005 


5 


13.2 


13.3 


11.4 , 


8,6 


<.005 


6 


42.8 


41.4 


34.4 

i • ‘ 


29.0 


<•005 



**Levels of significance for this and following tables are from analyses 
of covariance, which, along with tests used to determine significance 
of differences between means, are found in the respective technical 
reports, - 
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appeared to be associated with most of the gains in cognitive objectives. 

The addition of the mobile classroom did not substantially increase the 
number of cognitive objectives achieved by the children. The conclusion 
based on analysis of APT data was that the television program provides the 
basic information for the children, while the home visitors working with 
the parents and children effectively reinforce the program's cognitive 
objectives. A description of the APT analyses and results is included as 
Technical Report No. 14. 

/ The Peabody Picture Vocabulary Test (PPVT) was administered to all 

* children. The PPVT post-test raw scores, along with age in months, were 
j used as covariates in order to permit partial adjusting of certain sub - 
\ ^ * test means according to differences in age and intelligence in the dif- 
ferent treatment groups. The mean measured IQ's for different treatment 
■l* groups ranged from 104 in the TV-HV-MC group to 97 in the TV only group] 

# therefore, the small covariance adjustment would be downward for the 
.TV-HV-MC group and upward for the TV only group. Further explanation is 
reported in Technical Report No. 13. 

- ' 1 i * 1 < 

V Language Development, A second category of objectives for the Appalachia 
*' preschool Education Program was language development. Language wp defined 
, operationally as responses to the Illinois Test of Psycholinguistic Abilities 
(itpA)j the use cf this instrument permitted comparisons with national norms , 
*. A brief description of each of the 10 subtests is given in Table 6, and more 
*. • comprehensive descriptions can be found in Technical Report No. IS, 

The ‘results from the analysis of ITPA data were about the same as those 
from the second field test year, and that was only moderately positive for 
the APEP. The order of differences in means was, in most cases, favorable,^ 
to APEP, but the results from only three of ten subtests were significant.. 

Participation on the mobile classroom contributed to the difference in 
ability to describe objects verbally indicated by Subtest 5 results. All 
groups except the TV only scored near or above national norms on all ITPA 
subtests except Subtest 5, and all groups were below national norms on this 
subtest which measures expressive language ability. Since participation in 
the activities of the mobile classroom tended to improve children's scores 
on this subtest, it appears plausible that the mobile classroom is meeting 
a real need among Appalachian children. 



a As indicated in Technical Report No. 15, an analysis of variance of 
the ITPA data indicated significant differences (p<. 05) across treatment 
groups on all subtests and the total, and the pattern of differences was 
the same as those presented in Table 6, The covariates were apparently 
masking treatment differences in the analysis of covariance reported in 
Table 6, 
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Psychomotor , The Marianne frostig Test of Visual Perception was used to 
measure development in the areas of motor coordination and perceptual 
learning tasks. As shown in Table 7, the groups which received the Appa- 
lachia Preschool Education Program achieved significantly higher scores 
than the control group on four of the five subtests of the Prostig, as well 
as the total score. The proper names of the subtests on which the APEP 
groups achieved higher scores were (1) Eye-Motor Coordination, (3) Con- 
stancy of Shape, (4) Position in Space, and (5) Spatial Relationships. The 
differences on one subtest, Figure-Ground (2), were not statistically sig- 
nificant. 

The pattern of results from the analyses of Frc tig scores indicates 
that the television program was having a major effect on eye-motor coordi- 
nation, the ability to recognize shape constancy, and the ability to con- 
serve patterns after spatial rotations. The paraprofessional intervention 
was associated with learning in the area of same -different discrimination 
in terms of spatial rotation. In general, the results indicated that the 
television program was having a broad effect on children's perceptual motor 
development, and that the use of the mobile classroom did little to improve 
children's performance in the areas measured by the Frostig, 

The analyses of the Frostig data indicated that the AEL program was 
having a positive effect on the motor coordination and perceptual learning 
ability of Appalachian children, and this effect most likely was due to 
the emphasis on artistic and graphic activities which occurred throughout 
the program. Further details of the Frostig analyses are given in Technical 
Report No , 16 . 



Combination of Test Results . A factor analysis was performed with the 20 
subtests of the APT, PPVT, ITPA, and Frostig entered as variables. The 
purpose was to find whether the test scores would group into a combination 
which would have special meaning or identity and whether certain treatment 
groups would achieve higher factor scores than other groups. 

As reported in Technical Report No, 17, most of the variance in the 
20 subtests was explained by three factors. The first factor contained 
most of the ITPA subtests along with the PPVT Raw Score and was therefore 
given the label "visual identification, " An analysis of variance of factor 
scores indicated that there was no significant differences among the treat- 
ment group oh this factor. 

The second factor was composed primarily of the Frostig subtests along 
with ITPA Subtest 8 and was called "psychomotor." The differences in mean 
factor scores were significant (p<,005) for this factor, and the sequence 
of means from highest to lowest was TV-HV, TV-HV-MC, TV only, and control, 
with the first two treatment groups having almost identical mean factor 
scores . 
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Table 6 



Mean Post-test Scores and Levels of Significance 
for the Illinois Test of Psycho linguistic 

Abilities by Treatment Group 



Subtest 


Des cription TV-HV-MC 


TV-HV 


TV only 


Control 


Significance 


i 


i — — 

Vocabulary and 
hearing level 


22,7 


21,9 


19,5 


18.8 


- 


2 


Ability to match 
from a sample 


16.0 


14.7 


13.9 


14.0 


- 


3 


Vocabulary auditory 
association 


18.0 


16.8 


16.0 


14.3 


- 


4 


Association and 
good stimuli 


16,7 


16.1 


15,5 


13.9 


- 


5 


Ability to describe 
objects verbally 


15.0 


13.2 


12.2 


14.1 


<,01 


6 


Vocabulary and ability 
to communicate 
gestures 


23,8 


21.8 


23.1 


ia.S 


<.05 


7 


Ability to make 
grammatical 
transformations 


14.5 


16.4 


13.3 


13,3 


- 


8 


Figure ground 
dis crimination 


16.7 


12,6 


14.2 


13,6 


- 


9 


Auditory recall 


19,4 


16.2 


18.2 


16,8 


- 


10 


Visual recall 


10,9 


9.6 


12.4 


10,1 


<.01 


Total fall subtests) 


173.4 


163.7 


158,4 


147,6 


- 
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Table 7 



Mean Post-test Scores and Levels of Significance 
for the Frostig Test by Treatment Group 



Subtest 


Description 


TV-HV-MC 


TV-HV 


TV only 


Control 


Significance 


1 


Hand-eye coordination 
in line drawing 


9.4 


8.5 


9.6 


7.2 


<.01 


2 


Figure ground . 
discrimination 


10.1 


11,0 


10.0 

4 


8.6 


- 


3 


Recognition of 
geometric shapes 


6.9 


, 6.0 


5.3 


3.8 


<.005 


4 


Discrimination of 
figure rotation 


4.2 


4.5 


3.4 


2.7 


<.05 


■5 


Analysis and repro- 
duction of simple 
patterns 


2.7 


2.4 


1.8 


1.4 


<,005 


Total 


. Score (all subtests) 


33.2 


31,8 


29.8 


23.8 


<.01 
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The third factor was made up of the APT subtests and partly by four . 
other subtests and was called "vocabulary . ” The mean factor scores for 
treatment groups also were significantly different (p<. 001) , and the order 
of means from highest to lowest was TV-HV-MC , TV-HV, TV on 1 y , and control. 

The results of the factor analysis followed precisely the same pattern 
as the previously reported analysis of the separate tests. There were few 
differences on the language test , substantial differences on the psycho- 
motor, with the TV-HV-MC and TV-HV groups achieving more cognitive objec- 
tives than the TV only or control groups. The results of the factor analysis 
therefore were considered a validation of the separate tests and established 
the fact that the different test batteries were measuring different areas of 
performance of the children. 

Social Skills. A -preliminary attempt was made during the 1969-70 school 
year to measure social skills acquired by children in the TV-HV-MC group and 
in the TV-HV group. The measure of social skills development was replicated 
during the third field test year (1970-71), and the TV only group of chil- 
dren was added to the comparison. 

One of the original purposes for introducing the mobile classroom was 
that children would learn certain social skills, such as asking questions, 
responding to peers, and initiating statements. It was hypothesized that 
exposure to the mobile classroom (TV-HV-MC) would result in the development 
of social skills important to learning, in addition to the cognitive skills 
acquired by those children exposed only to the television program and home 
visitation by paraprofessionals (TV-HV) . 

The analysis of social skills among preschool children consisted of a 
systematic observation of interactions among groups of children and was 
considered an innovative evaluation technique based on interaction analysis 
procedures. Therefore, the method of observation as well as the results of 
the analysis was under study. The group receiving the mobile classroom 
(TV-HV-MC) did possess measurably greater social skills development at the 
end of the second field test year; however, the task itself did not produce 
the desired discrimination among individual children that was needed for an 
efficient comparison. Since actual treatment differences possibly were being 
masked by a task that failed to discriminate, a task was selected during the 
third field test year which hopefully required more group participation from 
all individuals. The task selected for the third field test year was 
directing a battery operated train on plastic tracks and erecting trees, 
buildings, animals, and people on a plastic mat. The second task did not 
appear to discriminate better than the first, but observed differences again 
indicated that children who participated on the mobile classroom possessed 
more constructive social skills. 

The design for the third year's measurement called for 36 randomly 
selected children from each of three treatment groups. Data were collected 



on 31 TV-HV-MC children, 26 TV-HV children, and 31 TV only children, for a 
total sample of 88. There were about equal numbers of males and females, 
and of 3-, 4-, and 5-year-olds. Data concerning the children's interactions 
were systematically coded using prearranged categories from videotapes of 
groups of from two to four children manipulating the train and other equip- 
ment. The data were analyzed through standard interaction analysis technique 

The analyses from the third field test year indicated that the TV-HV-MC 
children initiated more constructive statements than the TV-HV children, who 
in turn initiated more constructive statements than the TV only children. 

The TV-HV-MC group showed more enthusiasm, had the least inclination to 
withdraw from the task or to become distracted. The TV-HV children (no 
mobile classroom) were least inclined to stop working but in apparent contra- 
diction were most likely to become distracted. One explanation was that chi 1 
dren might stop working for reasons other than distraction. The TV only chil 
dren tended to withdraw from the group, either to work alone and/or for 
security. 

The TV only children tended to meet antagonism with antagonism and 
often initiated antagonistic behavior. The TV-HV children appeared to be 
more helpful than the TV-HV-MC children. 

The general pattern of differences, from greatest to least social skills 
development, was TV-HV-MC, TV-HV, TV only. The home visitor appeared to be 
of some effect in developing social skills and participation on the mobile 
classroom of more effect. The exceptions were that the TV-HV children were 
least inclined to stop working and appeared to be more helpful. 

The details of the procedures and analyses and a more complete descrip- 
tion of the results are given in Technical Report No. 7 for the second field 
test year and in Technical Report No. 18 for the third field test year. 

In conjunction with the social skills measure, a situation was arranged 
so that a measure of children's curiosity could be taken. A small room was 
furnished with familiar children's toys such as dolls and balls along with 
an unusual device which the children could manipulate to produce varied 
lighting and sound effects. Turning different knobs on the device caused 
lights to flicker or dim and produced unusual noises. A random sample of 
81 children from the APEP treatment groups were introduced to the room one 
at a time with a parent. The interaction of the child with different objects 
was coded each three seconds for 15 minutes , and the percent of time spent 
with the unfamiliar device was considered a measure of the child's curiosity 
or urge to learn. 

The children who had visited the mobile classroom in addition to other 
treatments (TV-HV-MC) were found to spend a greater percentage of their time 
(p<,001) interacting with the unfamiliar device than children who received 
only the home visitor and the instructional television program (TV-HV). 

Also, children who were visited by the paraprofessional each week in addition 
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to watching the television program (TV-HV) spent significantly more time 
with the unfamiliar device than children who only had access to the tele- 
vision program (TV only). The boys also were found to spend more time with 
the device than girls (p^.OOl), 

In other words, the home visitations were associated with intellectual 
curiosity or urge to learn, and attendance on a mobile classroom was asso- 
ciated with an additional degree of curiosity. The procedures and analyses 
are discussed in Technical Report No. 22, 

Parental Interest in Television Programs . A survey was conducted to measure 
parental response toward the following children's television programs: 
Misterogers, Captain Kangaroo, Romper Room, AEL’s Aro und th e Bend , and Sesame 
Street . The parents were asked to rate the programs according to those they 
liked best and least and indicate if their children watched the programs, 
enjoyed the programs, or learned from the programs, and whether they thought 
the programs were good for their children. The parents also were asked if 
they watched the programs with their children. 

The forms were mailed to a random sample of 300 parents whose children 
received different combinations of the APEP television program, home visita- 
tion, and mobile classroom experience; 210 replies were received, for a 70 
percent return rate. 

Parents of children in AEL's program rated the noncommercial black-and- 
white instructional television program produced by AEL as good as or better 
than other children's commercial television programs. The highest rating 
was given to Around the Bend by every group of parents and by 47 percent of 
all the parents responding , Sesame Street was second with 27 percent, followed 
by Captain Kangaroo with 22 percent, Misterogers with two percent, and Romper 
Room with one percent of the first place ratings. 

F* 

Eighty-nine percent of the parents reported that their children watched 
Around the Bend three or more times a week, and 80 percent of the parents 
reported watching the program with their children. Ninety percent of the 
parents thought their children learned from Around the Ben d and encouraged 
their children to watch the program, - 

The parents reported that the children watched the instruction tele- 
vision programs , that they themselves watched the programs, that both they 
and the children enjoyed the programs, and that they thought their children 
learned from watching the programs. Theserhighly positive results were 
regarded as an indicator of parental support for the APEP approach as well 
as parental acceptance of instructional television. Technical Report No, 

21 is a detailed presentation of these data. 



Comparison with Kindergarten Programs , As a result of a suggestion offered 
during a meeting with school officials in the Beck ley, W.Va. , field test 
area, a one-year comparison of the Appalachia Preschool Education Program 
with a standard kindergarten program was made* The comparison involved 66 
children in two kindergartens near the field test area, 111 5-year-old chil- 
dren in the AEL program, and a control group of 34 5-year-old children. 

Most of the AEL program children had been in the program for three years. 

The children were administered the Peabody Picture Vocabulary Test and 
three sections of the Appalachia Preschool Test (described in Technical 
Report No. 14). The latter was designed to measure achievement of objec- 
tives included in the APEP curriculum. In addition , socioeconomic data were 
collected from the parents,. , An analysis of these data (mentioned previously) 
indicated that there were no significant differences in socioeconomic level 
among the groups. There were initial differences in IQ, and PPVT post-test 
raw scores as well as age in months were used to make slight adjustments in 
the means on other subtests through an analysis of covariance. 

The analysis of the Appalachia Preschool Test data is presented in 
Technical Report No. 23, and Table 8 of this report shows the pattern of 
differences in achievement. The usual order of scores on the subtests of 
the Appalachia Preschool Test from highest to lowest was TV-HV-MC, TV-IIV, 
kindergarten, TV only, and control. The preliminary expectation by AEL 
staff was that the first three groups would score equally on the test of 
cognitive objectives. However, the TV-HV-MC and TV-HV groups actually 
scored significantly higher than the kindergarten group on some of the 
subtests , and at least equally well on all subtests, and higher than the 
control group on all of the subtests. The children who received only the 
TV program scored at the level of the control. 

The finding that certain APEP groups scored higher on measures of cog- 
nitive objectives than the kindergarten group is one of the most important 
of the evaluation. In a previous section, an analysis of cost data indicated 
that the AEL program could be operated for about one half the cost of a 
standard full day kindergarten program, when used by as many as 25,000 
children. The cognitive objectives on which the program and the APT were 
based were derived from a study of previous intervention programs and the 
characteristics of Appalachian preschool children. If these objectives are 
appropriate for 5-year-old children, then the APEP Is a more cost effective 
means of attaining them than a standard kindergarten program. 

Summary of Performance . The preceding section has examined evidence of 
changes in children's performance which appear to be associated with 
different components of the Appalachia Preschool Education Program. Table 
9 is a summary of the effects which were most obviously associated with 
either the instructional television program, the paraprofessional home 
visitor, or the mobile classroom. Note that the components are not inde- 
pendent and must function as a whole, although certain components tend to 
be associated with positive changes in performance. 



Table 8 



Mean Post-test Scores and Levels of Significance for the 
Appalachia Preschool Test and PPVT IQ by Treatment 
Group and Including Kindergarten Sample* 



APT Sub test 


TV-HV-MC 


TV-HV 


TV only 


Control 


Kindergarten 


Significance 


2 


44.6 


44.4 


39,0 


36.8 


42.0 


<.005 


5 


15.1 


16.4 


13.4 


11.1 


12.9 


<.005 


6 


49.2 


47.3 


40,8 


37.1 


46.6 


<.005 


PPVT I.Q. 

t--* 


102.9 


105.0 


99.8 


99.5 


103.8 


<.005 


N 


31 . 


49 


31 


34 


66 


----- 



*A11 children were in the 5-year-old age group. 




Table 9 



Summary of Contributions to Children's Performance of 
Instructional Television Program, Home Visitor, 
and Mobile Classroom 



Instructional Television 

Program Paraprofesslonal Home Visitor 



The television pro- 
gram provides the basic 
curriculum on which the 
other components depend. 
Although it effectively 
teaches a number of 
cognitive objectives 
without further rein- 
forcement, it is most 
effective in this area 
when operating in con- 
junction with the para- 
professional. 

Hie television pro- 
gram also significantly 
aids in perceptual - 
motor development by 
encouraging manual 
tasks such as drawing 
and cutting. 

Further effects 
of the television pro- 
gram are evident in 
reaching readiness 
skills such as the 
ability to recognize 
geometri c sh apes and 
to conserve relational 
pattern. 



The paraprofessional's 
main function is to reinforce 
the child's learning from the 
basic curriculum, This rein- 
forcement is done by working 
indirectly with the parent, 
and directly with the child. 

The effectiveness of the 
home visitor is evident in 
increased learning of cogni- 
tive objectives from the 
television program and in 
broad areas of increased 
language growth. 

The paraprofessional 
also facilitates perceptual 
development in terms of 
shape recognition and 
reading related skills. 




Mobile Classroom 



The primary function 
of the mobile facility 
is the development of 
the skills necessary 
for constructive social 
interaction. It meets 
the need of rural and 
isolated children for 
structured group inter- 
action. 

Data from the third 
year's program effort 
indicates that the van 
experience effectively 
teaches children to 
cooperate on group 
tasks and facilitates 
their social develop- 
ment. 

Additionally, the 
mobile facility pro- 
vides a stimulus to 
the child's curiosity 
and his overall urge 
to learn in novel 
environments. 

Finally the mobile 
facility encourages 
the child to express 
himself freely in a 
nonverbal manner and 
to interact more 
freely in a group 
setting- 
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Program Performance Pervasiveness 

Program performance pervasiveness is defined as the base for diffusing 
the Appalachia Preschool Education Program, i.e,, the number and kind of 
individuals who can be affected by operation of the program. 

The pervasiveness of a program under development, especially a program 
such as the Appalachia Preschool Education Program, may be very different 
when the program is made operational. The program is designed to operate 
on a regional basis encompassing- several school systems. The television 
lessons broadcast from the Oak Hill station reached homes over an eight- 
county area of southern West Virginia. For development purposes, however, 
the mobile classroom and home visitation coi^ponents were extended only to 
the number of youngsters required to develop the program and conduct suffi- 
cient evaluation of it during the field test years of 1968-1971, 

The program is considered to be a unified set of activities comprising 
television instruction, home visitation, and mobile classroom instruction. 

It is designed for optimal operation on a regional, statewide, or even multi- 
state basis. As explained previously, program performance pervasiveness is 
predicated on a population unit of 25,000 children in West Virginia, 



Description of West Virginia Population 

The population of West Virginia was used as a basis for estimation 
because the state is of adequate size to make the program feasible and 
because the program was designed for this type of terrain and population. 
Similar data might be derived for other regions i i Appalachia or elsewhere. 

On April 1, 1970, the population of West Virginia was 1 ,744,237 according 
to the IJ.S. Census Bureau (Bureau of Census, 1970). The same report showed 
that West Virginia had suffered a loss of 6,7 percent in population since 
1960. The 1970 Census reported that 61 percent of the people in West 
Virginia were living in rural areas (i.e. > communities of 2,500 or less), 
and that there were on the average 72 persons per square mile in the state. 

According to the West Virginia Education Association (1971) , 30 .6 per- 
cent of *the adults age 25 and over had completed high school in 1968. The 
median per capita income in West Virginia in 1970 was $2,610. 



Children of A^s 3, 4, and 5 

At the time of this writing, the exact .number of children of ages 3, 4, 
and 5 in the state in 1970 had not been tabulated by the U .S .Census Bureau. 
However, the number of children of ages 0 through 4 and of ages 5 through 6 
had been tabulated, and the following estimates were projected from those 
tabulations (AEL, 1971) . On April 1, 1970 ,therewas an estimated 28,841 



3-year-olds, 29,747 4-year-clds, and 30,707 5-year-olds, for a total 3-, 4-, 
and 5-year-old population of 89,295. Of this number, the children living 
in rural areas included 17,593 3-year-olds „ 18,146 4-year-olds, and 18,731 
5-year-olds, or a total of 54,470. 

Estimates of the numbers of 3-, 4-, and ‘S-year-old children by county 
.varied from 3,438 in Kanawha County and 2,652 in McDowell County to 332 in 
Pendleton County and 228 in Wirt County. 

To provide the Preschool Education Program, for all the 3-, 4-, and 
5-year-old children in the state of West Virginia (89,295) would require 
about three and one half times the program "effort described in the Program 
Cost Analysis section of this report for 25,000 children. The operational 
cost of the AEL program for all the 3-, 4- ,* and 5 -year-old children in rural 
areas in the state would be $13.6 million (54,470 rural children x $250. 3a 
per child). By comparison, a standard kindergarten program would cost 
approximately $27 million (54,470 children tx,.$496 per child). 

The operational cost of the APEP for ijI- Vyear-old children in West 
Virginia would be $7.7 million (30,707 children x $250.33 per child) compared 
to $15,2 million for a standard kindergarten program (30,707 children x $496 
per child). To provide the APEP for all rural • 5-year-old children (18,731) 
would require less than the unit cost for- 25,000 children. 

In summary, the program pervasiveness and program cost studies indicate 
that the Appalachia Preschool Education Pro|rafn is an economical alternative 
to other early childhood education programs* tv <* . 

I *• « : i 

Evaluation Synthesis 

The purpose of the summative, evaluation of the Appalachia Preschool 
Education Program' was to provide information from which decisions could be 
made regarding the viability of the program as an alternative toother pro- 
cedures for early. childhood education. The 'following statements summarize 
the results of three years of evaluation, from 1968 to 1971. 



Cognition A 

. ’ A - * • . . 

Scores on a test of cognitive objectives favored children who received 
the Appalachia Preschool Education Program. The interaction of the para- 
profossional home visitor with parents and children was associated with 
substantial gains in the number of cognitive objectives achieved. 



Language 

A trend toward increased language development was observed for APEP 
children as opposed to children in comparison groups 



APEP children 



significantly higher scores on measures of the ability to describe objects 
verbally and to communicate by means of manual expression. 



Psychomotor 

Children who participated in the Appalachia Preschool Education Program 
scored significantly higher than comparison groups on the following traits: 
hand-eye coordination, recognition of geometric shapes, discrimination of 
figure rotation, and analysis and reproduction of simple patterns. The APEP 
children were found to have achieved a significantly higher level of motor 
coordination and perceptual learning ability than children who did not 
receive any program. These differences were considered due to the emphasis 
on artistic and graphic activities throughout the three years of APEP develop- 
ment . 



' Social Skills 

Children who participated in the mobile classroom activities gave indi- 
cations of having developed more constructive social skills than children 
who received only the home visitor and television program, and that group 
in turn was more socially constructive than children who only watched the 
television program. The pattern of differences in social skills development 
also was found on a measure of children's curiosity. 



Interest 



A random sample of 210 parents with children in AEL's program rated 
children's television programs on general appeal,, A first place rating was 
recorded by 47 percent of the parents for AEL's Around the Bend , 27 percent 
for Sesame Street, 22 percent for Captain Kangaroo , two percent for Misterogers , 
and one percent for Romper Room . Around the Bend was not in color. 

Eighty-nine percent of the parents reported that their children watched 
Around the Bend three or more times a week, and 80 percent of the parents 
said they watched the programs with their children. „ .Parents and children 
were encouraged to watch Around the Bend by home visitors: however, parents 
of children who were not exposed to either the home visitor or the mobile 
classroom rated AEL's program as high as those with access to those compo- 
nents . 

Kindergarten Comparison 

Children in the AEL program reached significantly more cognitive objec- 
tives than children in a kindergarten program in the same area, and both 

groups scored significantly higher on a test of cognitive objectives than 
children who received no treatment, , 
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Required Effort for 25,000 Children 

Eight professional and three support staff would be required for pro- 
duction of curriculim materials including television lessons. 

Field personnel requirements include one certified teacher and one 
aide for each 150 children, and one paraprofessional home visitor for each 
57 . 5 children. 

Based on AEL experience during three years of field testing, the pro- 
gram can be delivered to 25,000 children for an operational cost of $250.33 
per child. An additional capital outlay cost of $21.68 per child (amortized 
over five years) would be required. 

The cost for operating the APEP is about one half the operational cost 
of educating a child in a conventional classroom, and the required capital 
outlay is less than one seventh of the cost for conventional classroom 
education. 



Hooper, Frank H. and William H. Marshall. The Initial Phase of a Preschool 
Development Project (Charleston, W.Va, : Appalachia Educational Labora- 

tory, Inc., 1968). 

"Interpolation of 1971 Census Data to Obtain Number of Three, Four, and 

Five Year Old Children Living in Rural Areas" (Charleston, W.Va.: Appa- 

lachia Educational Laboratory, Inc., 1971). 

1970 Census of Population - West Virginia (Advance Report) (Washington, D.C.: 
Bureau of the Census, 1970). 

Ranking of the Counties (Charleston, W.Va.: West Virginia Education Associa- 

tion, April, 1971). 

Stufflebeam, Daniel L. et_. aU Educational Evaluation and Decision Making 
(Itasca, 111.: F. E. Peacock Publishers, Inc., 1971). 
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List of Available and Proposed Publications Pertaining 
to the Appalachia Preschool Education Program 



Available Evaluation Reports 



Evaluation Reports Early Childhood Education Program, 1969 Field Test, Appa- 
lachia Educational Laboratory, Inc., Charleston, W.Va. , ED 041 626. 

Evaluation Report: Early Childhood Education Program, 1969-70 Field Test, 

Appalachia Educational Laboratory, Inc., Charleston, W.Va., 1S71. 

Evaluation Report: Early Childhood Education Program, 1969-70 Field Test. 

Summary Report, ED 052 837. 

Demographic and Socio-Economic Data of the Beckley, W.Va., Area and 1968-1970 
Developmental Costs of the Early Childhood Education Field Study. Tech- 
nical Report No. 1, ED 052 832. 

Analysis of Intelligence Scores. Technical Report No. 2, ED 052 838. 

Attainment of Cognitive Objectives. Technical Report No. 3, ED 052 833. 

Detailed Analysis of Language Development of Preschool Children in BCE Pro- 
gram, Technical Report No, 4, ED 052 834. 

Analysis of Visual Perception of Children in the Early Childhood Education 
Program (Results of the Marianne Frostig Developmental Test of Visual 
Perception). Technical Report No. 5, ED 052 839. 

Factor Analysis of the Early Childhood Education Test Data. Technical Report 
No. 6, ED 052 840. 

, Social Skills Development in the Early Childhood Education Project. Technical 
Report No. 7, ED 052 835, 

Results of Parent and Student Reaction Questionnaire, Technical Report No. 

8, ED 052 836. 

Analysis of Children's Reactions to AEL's Preschool Television Program. 
Technical Report No. 9, ED 052 841, t ... 

A Comparison of Parents * /'AttitudesiTwafy.'AEL'isr ^Around the Bend and Other 
Children's Television Programs. Technical Report No, 10, ED 052 842, 



Summative Evaluation of the Appalachia Preschool Education Program, Appala- 
chia Educational Laboratory , Inc . , Charleston, W.Va. , December, , 1971, 




Demographic and Socioeconomic Data of the Beckley, W.Va.,, Area mid 1968- 
1971 Development Costs of AEL's Preschool Education Field Study. 

Technical Report No. 11. 

Analysis of Intelligence Scores. Technical Report No. 13. 

Attainment of Cognitive Objectives. Technical Report No. 14. 

Detailed Analysis of the Language Development of Children in AEL’s Pres hool 
Education Program. Technical Report No. 15. 

Analysis of Visual Perception of Children in the Appalachia. Preschool Educa- 
tion Program. Technical Report. No, 16. 

Factor Analysis of the Appalachia Preschool Education Program Test Data, 
Technical Report No, 17. 

Analysis of Social Skills Development in the Appalachia Preschool Education 
Program. Technical Report No. 18, 

A Comparison of Parents* Attitudes Toward AEL's Around the Bend and Other 
Children’s Television Programs. Technical Report No. 21. 

Measuring Children's Curiosity, Technical Report No, 22, 

A Comparison of AEL's Preschool Education Program with Standard Kindergar- 
ten Programs. Technical Report No. 23. 

Other Available Publications 

Marshall, H, William and Frank II. Hooper, The Initial Phase of a Preschool 
Curriculum Development Project, (Final Report) Appalachia Educational 
Laboratory, Charleston, W.Va,, August, 1968. 

Appalachia Educational Laboratory: Early Childhood Education, Career 

Decision-Making, Educational Cooperatives, November, 1971. (Brochure) 

Alford, Roy, The Appalachia Preschool Education Program: A Home-Oriented 

Approach, ED 052 865, 



Alford, Roy , Teaching Mathematical Concepts to Rural Pres chop 1 Children 
Through a Home-Oriented Program, Doctoral Dissertation , Appalachia 
Educational Laboratory, Charleston, W.Va,, June, 1970. 




j Proposed Publications for 1972 

(not exact titles) 

Implementation Guide (a description of the program and directions for 
beginning implementation)”. 

Materials Preparation Manual (directions for producing videotapes, 
home use materials, and mobile classroom materials suitable for use in 
operation of program). 

Field Operati ons Manual (describes training procedures and design for 
operation field activities). 

Home Visitors Handbook (describes materials, methods, and activities 
and cites further references) . 

Mobile Classroom Teacher Ha ndbook (describes materials, methods, and 
activities and cites further references) . 

Curriculum Guide (a list of suggested behavioral objectives for chil- 
dren enrolled in program) . 

Personnel Training Guide ^specific procedures and activities to be 
used in preservice and inservice training of field personnel) . 

Evaluation Manual (suggested procedures and instruments to be used in 
assessing the effectiveness of the program and providing feedback for 



program improvement) . 
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The Appalachia Educational Laboratory defines educational development 
as the systematic process of creating and diffusing alternative products 
that will contribute to the improvement of educational practices. This in- 
creases requirements of the process beyond earlier models providing for 
product development only. 



The rationale for this position is that, in most instances, the suc- 
cessful implementation of educational products requires more than the general 
dissemination of information about the product. To break down the historical 
educational pattern of teachers and classrooms and children by 25s and to 
substitute for it a model whose structures and practices are built increas- 
ingly on knowledge, whose central concerns are for continuous improvement, 
and whose basic posture is one of accountability, requires changes in skills, 
attitudes, and motives of educational practitioners. While ABL firmly sup- 
ports the position that the only reliable approach to the improvement of 
these qualities is through the creation and utilization of thoroughly tested 
..educational products , it also holds the belief that there must be substan- 
tial involvement by potential users in the planning and development of the 
products if systematic adoption is to be effected. a 






ConsistentiWi'tlf 1 this rationale, the Laboratory employs a model for edu- 
cational development consisting of seven stages of work with both functions 
of product development and product diffusion carried bh simultaneously but 
with differing emphasis at various stages . The model and the .interrelation- 



ships between product development and product diffusion are shown in Figure 



1 of this attachment. 



The stages of educational development disp 

follows:-: .. •. {AX'.- ' 



in 



X are as 






Stage 1 : Needs Assessment •. : 

Stage 2 : . Feasibility Analysis V . 

Stage 3:-' Program' Planning , ■ , 

Stage 4: Product Design 'and Engineering 

Stage 5: Field Testing 

Stage 6 : Operatibna.1 Testing 
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ndicated in figure 1 , the tir^ Tcnree sta^s eqnrain -the series of 
decisions which make up Program Planning Strategy. { The importance of the 
function of diffusion in Program Planning, Strategy is illustrated by the 
fact that ^ to- plan a program tojdevelopy a >prbductawhich^a^ 
problem: of no, concern to.i educational pr a ef iti^ 

mehtedlwbuibXe a wasted effort. 
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Stage 1 



Stage 2 



Stage 3 




The fourth, fifth, and sixth stages contain the decisions which make up 
the Product Development Strategy. Here the product development function is 
of more importance than the product diffusion function, but some diffusion 
activities are required during these stages, particularly in Stage 6, 

The seventh stage. Dissemination and Implementation, contains a contin- 
uation and culmination of the steps taken to carry completed outputs for- 
ward to produce the intended outcomes with the specified target populations 
and constitutes the Diffusion and Implementation Strategy,. Here the product 
diffusion function is of major importance. - 

Internal to each stage is a series of activities, outputs, and criteria 
for advancing to the next stage of work. Any development efforts not meet- 
ing the specified advancement criterion in Stages 2 through 6 are recycled 
until the criteria are satisfied, or alternatively, a decision is made to 
abort the effort. This recycling process, with resulting' improved perfor- 
mance, is fundamental in educational development and is apparent in the 
statement of activities in each stage of development. 

Stage 1: Needs Assessment 

The purpose of the needs assessment stage is to determine the priority 
of educational needs of the region which are appropriate for solution by 
educational development activities. 

1) Collection and analysis of regional educational 
and demographic data* 2) assessment of lay and pro- 
fessional perceptions' of regional educational needs; 
and 3) ranking of educational needs in priority 
order based upon significance and probability of 
implementation of solutions, 

Ij An information base to assist in decision-making 
relative to appropriate attacks on educational prob- 
lems; and 23 a priority list of regional educational 
needs whose solutions can be implemented. 

High probability that the priority needs listed cor- 
respond to actual needs and can be solved by educa- 
tional development products . 

Stage 2*. Feasibility Analysis 

The purpose of feasibility analysis is to determine if it is feasible 
for the Laboratory to plan a development program with objectives to meet a 
specific educational need. 
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Activities : 



O utputs : 



Advancement 
Criterion : 



Activities : 



1} Selection of a specific regional need; 2) deter- 
mination of general outcomes to be achieved by the 
products of the development program; 3) application 
of the following feasibility, criteria: 



a. Are resources available or obtainable to 
mount the necessary developmental effort? 



Does the necessary knowledge exist to 
develop an acceptable (to achieve the 
general outcomes) solution? 



c. 



Is the program to be deve 
with the mission of the La 



consistent 
? 



d. Is the need of enough importance to 
make implementation* probable? 



e. Will the estimated cost of the product 
not be prohibitive to the consumer? 



and 4) determination of the' objectives of a program 
which is to be 



Outputs 



1) Evidence of the selected educational need; 2) 
.statement of general outcomes desired as a solu- 
tion to the need; 3) comprehensive documentation 
that a program can be successfully developed and 
implemented by. the Laboratory; and 4) a statement 
of objectives to be achieved .by the program to be 



-Advancement High probability that an educational development 
Criterion: program to achieve specified objectives could be 

supported by anticipated Laboratory resources. 



Stage 5: Program Planning 



the purpose of the program planning stage is to decide upon a 
program and prepare a plan for developing the product. 






Activities : 



1) Definition of alternative programs with firm 
estimates of associated costs-to-benefits for 
target populations; 2) estimation of development 
costs for each alternative;,* 3) determination of 
most appropriate program; 4) ^preparation of pro- 
gram work plan; and 5) determination of level 
of acceptability of proposed products through 
contacts and involvement of regional constitu- 



encies 



O 
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Outputs : 



1) Documentation of cost-benefit ratios for 
alternative solutions; 2) documentation of esti- 
mated development costs for each alternative; 

3) documented basic program plan detailing prob- 
lem to be solved, specific product objectives, 
product development and diffusion strategies, 
and work plan with costs by development stages; 
and 4) documentation of constituencies* recep- 
tivity to program. 



Advancement 

Criteria: 



1) High probability that the Iprogram as planned 
will achieve the objectives to ameliorate the 
need; 2) the approval of the iBasic Program Plan 
by the U.S. Office of Education;- and 3) evidence 
that regional constituencies are receptive to tin 
proposed solutions. 



Stage 4: Product Design and Engineering 



The purpose of the product design and engineering stage is to design, 
construct, preliminary test, and redesign the product. 



Activities : 



1) Preparation of specifications for the product; 

2) preparation of the design of the product; 3) 
preparation of procedures and, instruments for pro- 
duct evaluation; 4) consultation with and support 
of regional constituencies regarding features 

and possibilities of product design; 5) construc- 
tion of prototype and/or elements in limited, 
simulated environment; and 6) redesign and re- 
construction of prototype to eliminate observable 
deficiencies . 



Outputs ; 



1) Prototype product; 2) evaluation procedures and 
instruments; and 3) evidence on consistency of pro- 
duct with potential users* expectations. i 



Advancement 
Criteria : 



1) Documented high efficiency of product in produc- 
ing specified outcomes in limited, simulated 
environment; and 2) evidence that the product is 
consistent with potential users* needs and a. 
ities . 



Stage 5: Field Testing 



The purpose of the field testing stage is to test the product, under 
Laboratory control, with a sub-set of the target population in a setting 
approximating a typical educational environment to ascertain whether the 
product can produce stated outcomes. r ' 
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Activities : 


1) Identification and establishment of relation- 
ships with constituencies for field test site; 

2) placement of product in operational mode;, 3) 
testing of product; 4) product revision based 
upon field tSst data; and 5) provision of full 
information on field test to regional constitu- 
encies , 


Outputs : 


1) Documented field test results of the use of 
the product; and 2) revised product. 


Advancement 

Criteria: 


Ij Evidence that the product meets specifications 
and high probability that it will produce speci- 
fied outcomes in an operational test; and 2) 
^idence of interest in the product on the part 
of regional constituencies. 



Stage 6: Operational Testing 

The purpose of the operational testing stage is to test the product , 
with a minimum of Laboratory control, in a typical educational environment 
to ascertain if the product can produce stated outcomes in the target 



population. 




Activities : 


1) Identification and establishment of relation- 
ships with constituencies for operational test 
sites; 2) placement of product in operational 
mode; 3) test of product; 4) product revision 
based upon operational test data; 5) provision 
of opportunity for site visits, full information 
on availability of product to regional constitu- 
encies; 6) production of materials dealing with 
problems unique to implementation; 7) exploration 
of possiblj relationships with regional agencies , 
which could serve as linkages in implementation; 
and 8) exploration of the possibility of market- 
ing through commercial publishers or manufacturers 
and other means. ...••♦«- 


Outputs : 


1) Documented evidence of the results of the oper- 
ational test; 2) a revised and tested product; and 
3) readiness among regional constituencies for 
widespread implementation of the product. 


Advancement 

Criteria: 


1) Evidence, that product objectives are met at an 
acceptable level* and 2] product in a form to be 
broadly disseminated and implemented. 



Stage 7: Dissemination and Implementatio n 

ThP nm-nose of the dissemination and implementation stage is to achieve 
widespread implementation of the product by capitalizing upon the readiness 
for adoption bv regional constituencies built during earlier stages. 



Activities : 



1) Completion of agreements for marketing through 
commercial publishers or manufacturers/ and other 
means; 2) activation of institutional linkages 
to advance implementation; 3) provision for infor- 
mation on necessary supporting systems for imple- 
mentation* 4) provision of full information on the 
product* and costs to implement to regional con- 
stituencies; and 5) maintenance of a record of 
product adoption and user satisfaction. 



Output : 



Reliable , proven product widely adopted 
mented- 



and imple- 



There are wide variations in requirements for resources at different 
stages of development. Resource requirements are minimal through the f it. t 

stages ;°accelerate sharply through the stages » d 

engineering, field testing, and operational testing; and then taper sharp!/ 

during dissemination and implementation. 
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